-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Wait for #2794, #2795][Graph] add inplace setting through layer property #2796
base: main
Are you sure you want to change the base?
Conversation
The method of setting the inplace-type has been redefined. The reason why inplace processing becomes complicated is that since a multi-out layer shares output variables, so it needs to be considered whether or not inplace can be performed. To simplify the problem, the layers that can perform inplace even after the multi-out layer are only no-operation layers(no-op layers). These no-op layers include identity, reshape, and flatten layers. For other layers, even if they support inplace, they cannot perform inplace when there is a multi-out layer in front of them. Note that because no-op layers connected with multi-out layer share memory with the multi-out layer, so they have the same properties as the multi-out layer. This is expressed as RESTRICTING in our script. Based on these definitions, I've redesigned the method of setting inplace type. 1. By default, initialize the inplace type for each layer. If supportInPlace is true, it will be initialized as NON_RESTRICTING; otherwise, it will be initialized as NONE. 2. However, not all layers are initialized like this. For multi-out layers or no-op layers, if supportInPlace is true, they will be initialized as RESTRICTING types(However, the no-op layer will be changed to a non-restricting type if that is not connected with the multi-out layer). 3. After initialization, confirm the input connections from the network_graph.cpp to determine the final inplace type. It's clearer to see the source code for this part. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Seungbaek Hong <[email protected]>
- The variables with the same meaning are written as "in_place" or "is_inplace" by the script, so I unified it to use the term "is_inplace". - Some layer's finalize function includes code that determines whether or not to support in-place depending on the tensor type. However, this code does not work. The reason it seems like this code is working is because there is a similar purpose of code at the top of the `canExecuteInPlace` function within the `network_graph.cpp` and that code works. It is meaningless to determine whether or not to support in-place within the `finalize` function because the `canExecuteInPlace` function, which decides how InPlace will behave, is called before the `finalize` function. The canExecuteInPlace function is called during `compile` while the finalize function is called during `initialize` after `compile`. Therefore, setting supportInplace inside the finalize function does not work. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Seungbaek Hong <[email protected]>
📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2796. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@baek2sm, 💯 All CI checkers are successfully verified. Thanks.
- now you can set in-place flag through layer property of tensor operation layers. - rename "initializeInPlaceType" func to "initializeInPlace". now "is_inplace" property is set in that function, too. - in some layers, support_backwarding flag may be changed by the in-place setting. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Seungbaek Hong <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@baek2sm, 💯 All CI checkers are successfully verified. Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With discussions made offline today, lgtm!
operation layers.
"is_inplace" property is set in that function, too.
in-place setting.
Self evaluation:
Signed-off-by: Seungbaek Hong [email protected]